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Period for Reply 



A SHORTENED STATUTORY PERIOD FOR REPLY IS SET TO EXPIRE 3 MONTH(S) OR THIRTY (30) DAYS, 

WHICHEVER IS LONGER, FROM THE MAILING DATE OF THIS COMMUNICATION. 

- Extensions of time may be available under the provisions of 37 CFR 1 .1 36(a). In no event, however, may a reply be timely filed 
after SIX (6) MONTHS from the mailing date of this communication. 

- If NO period for reply is specified above, the maximum statutory period will apply and will expire SIX (6) MONTHS from the mailing date of this communication. 

- Failure to reply within the set or extended period for reply will, by statute, cause the application to become ABANDONED (35 U.S.C. § 1 33). 
Any reply received by the Office later than three months after the mailing date of this communication, even if timely filed, may reduce any 
earned patent term adjustment. See 37 CFR 1.704(b). 

Status 

1)^ Responsive to communication(s) filed on 11 September 2007 . 
2a)D This action is FINAL. 2b)K This action is non-final. 

3) D Since this application is in condition for allowance except for formal matters, prosecution as to the merits is 

closed in accordance with the practice under Ex parte Quayle, 1935 CD. 11, 453 O.G. 213. 

Disposition of Claims 

4) E3 Claim(s) 19-32 is/are pending in the application. 

4a) Of the above claim(s) is/are withdrawn from consideration. 

5) D Claim(s) is/are allowed. 

6) 03 Claim(s) 19-32 is/are rejected. 

7) D Claim(s) is/are objected to. 

8) D Claim(s) are subject to restriction and/or election requirement. 

Application Papers 

9) [3 The specification is objected to by the Examiner. 

10) S The drawing(s) filed on 06 December 2005 is/are; a)S accepted or b)D objected to by the Examiner. 

Applicant may not request that any objection to the drawing(s) be held in abeyance. See 37 CFR 1 .85(a). 
Replacement drawing sheet(s) including the correction is required if the drawing(s) is objected to. See 37 CFR 1.121(d). 

1 1) D The oath or declaration is objected to by the Examiner. Note the attached Office Action or form PTO-152. 

Priority under 35 U.S.C. § 119 

12) E3 Acknowledgment is made of a claim for foreign priority under 35 U.S.C. § 1 19(a)-(d) or (0. 
aM AH b)D Some * c)D None of: 

1 Certified copies of the priority documents have been received. 

2. Q Certified copies of the priority documents have been received in Application No. . 

3. D Copies of the certified copies of the priority documents have been received in this National Stage 

application from the International Bureau (PCT Rule 17.2(a)). 
* See the attached detailed Office action for a list of the certified copies not received. 
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DETAILED ACTION 

1 . This action is responsive to communications: Application filed December 06, 2005 and 
Preliminary Amendment filed 09/1 1/2007. 

2. Claims 19-32 are pending in the case. Claims 19 and 25 are independent claims. 

Specification 

3. The title of the invention is not descriptive. A new title is required that is clearly 
indicative of the invention to which the claims are directed. 

The following title is suggested: Displaying Audio Data with Photo Data. 

Claim Objections 

4. Claims 23 and 3 1 are objected to under 37 CFR 1 .75(c), as being of improper dependent 
form for failing to further limit the subject matter of a previous claim. Applicant is required to 
cancel the claim(s), or amend the claim(s) to place the claim(s) in proper dependent form, or 
rewrite the claim(s) in independent form. 

Claim Rejections - 35 USC § 102 

5. The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the 
basis for the rejections under this section made in this Office action: 

A person shall be entitled to a patent unless - 

(e) the invention was described in (1) an application for patent, published under section 122(b), by another filed 
in the United States before the invention by the applicant for patent or (2) a patent granted on an application for 
patent by another filed in the United States before the invention by the applicant for patent, except that an 
international application filed under the treaty defined in section 351(a) shall have the effects for purposes of this 
subsection of an application filed in the United States only if the international application designated the United 
States and was published under Article 21(2) of such treaty in the English language. 

6. Claims 19,20,22-26,28 and 30-32 are rejected under 35 U.S.C. 102(e) as being 
anticipated by Bergman (US006564263B1, 5/13/2003). 
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Regarding independent claim(s) 19, 23, Bergman teaches: 

checking whether an asset for display (multimedia content) is comprised of one or more 
photo data and one or more audio data (col. 19, 11. 54-58, checks to see what the content is 
comprised, which in the normal course operation includes photo, image col. 8, 11. 28-36, and 
audio, col. 8, 11. 37-39); 

extracting reference information needed for displaying the photo data and the one or 
more audio data from the asset; (description scheme, col. 19, 11. 54-55); 

displaying the photo data and the one or more audio data, using the extracted reference 
information, based on a meta data including time information indicating time during which each 
piece of the photo data is displayed while the audio data is being provided (multimedia is 
displayed, col. 7, line 59, the multimedia is generated/synthesized according to (based on) intra- 
object relationships, col. 20, 11. 34-47, which includes a position and a time when the photo data 
is displayed (col. 6, 11. 53-56).) 

Regarding independent claim(s) 25, 31, Bergman teaches: 

displaying the asset using the generated metadata (multimedia is displayed, col. 7, line 
59, the multimedia is generated/synthesized according to (based on) intra-object relationships or 
meta-data, col. 20, 11. 34-47) 

the asset is comprised of audio, (col. 8, 11. 37-39) and one or more photo data (image, col. 
8, 11. 28-36) and is displayed by use of a asset corresponding to the reference information, 
wherein, the reproduction information comprises information on a time when the text data is 
displayed (metadata includes a time when the text data is displayed, col. 6, 11. 53-56). Inherently, 
this metadata must have been generated. 
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Regarding dependent claim(s) 20, 26, Bergman teaches a markup language (col. 14, 11. 3-29). 
Regarding dependent claim(s) 22, 30, Bergman teaches identification information to identify 
the asset (annotations, col. 8, 11. 55-58). 

Regarding dependent claim(s) 24, 32, Bergman teaches information on an attribute of an asset 
(annotations, col. 8, 11. 55-58). 

Regarding dependent claim(s) 28, Bergman teaches the elements are defined by a schema 
(scheme, col. 18, 11.63-67). 

Claim Rejections - 35 USC § 103 

7. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in 
section 102 of this title, if the differences between the subject matter sought to be patented and the prior art are 
such that the subject matter as a whole would have been obvious at the time the invention was made to a person 
having ordinary skill in the art to which said subject matter pertains. Patentability shall not be negatived by the 
manner in which the invention was made. 

8. This application currently names joint inventors. In considering patentability of the 
claims under 35 U.S.C. 103(a), the examiner presumes that the subject matter of the various 
claims was commonly owned at the time any inventions covered therein were made absent any 
evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1 .56 to point out 
the inventor and invention dates of each claim that was not commonly owned at the time a later 
invention was made in order for the examiner to consider the applicability of 35 U.S.C. 103(c) 
and potential 35 U.S.C. 102(e), (f) or (g) prior art under 35 U.S.C. 103(a). 

9. Claim 29 is rejected under 35 U.S.C. 103(a) as being unpatentable over Bergman. 
Regarding dependent claim(s) 29, Bergman does not give the elements the exact names in the 
claims. However, the differences in the claimed invention are only found in the nonfunctional 
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descriptive material and are not functionally involved in the steps recited. All the steps of the 
function would be performed the same way regardless of whether or not the name of the element 
is "AudioWithStill" or anything else (for example "Audlmg"). Thus, this descriptive material 
will not distinguish the claimed invention from the prior art in terms of patentability, see In re 
Gulack, 703 F.2d 1381, 1385, 217 USPQ 401, 404 (Fed. Cir. 1983); In re Lowry, 32 F.3d 1579, 
32 USPQ2d 1031 (Fed. Cir. 1994). Therefore, it would have been obvious to a person of 
ordinary skill in the art at the time the invention was made to call the elements any name 
(including "AudioWithStill") because such data does not functionally relate to the steps in the 
method claimed and because the subjective interpretation of the data does not patentably 
distinguish the claimed invention. 

10. Claims 21 and 27are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Bergman as applied to claims 18 and 25above, and further in view of Applicant's Admitted 
Prior Art. 

Regarding dependent claim(s) 21, 27, Bergman teaches the meta-data is described according to 
MPEG-7, a known standard (col. 14, 11. 4-8). Bergman uses this standard rather then MPV. 
Applicant admits that MPV was a known standard (para. 3). It would have been obvious to one 
of ordinary skill in the art at the time of the invention to substitute the MPV standard for the 
MPEG-7 standard because the substitution of one known element for another would have yielded 
predictable results (multimedia is still displayed according to the meta-data) to one of ordinary 
skill in the art. 

Double Patenting 

1 1 . The nonstatutory double patenting rejection is based on a judicially created doctrine 
grounded in public policy (a policy reflected in the statute) so as to prevent the unjustified or 
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improper timewise extension of the "right to exclude" granted by a patent and to prevent possible 
harassment by multiple assignees. A nonstatutory obviousness-type double patenting rejection 
is appropriate where the conflicting claims are not identical, but at least one examined 
application claim is not patentably distinct from the reference claim(s) because the examined 
application claim is either anticipated by, or would have been obvious over, the reference 
claim(s). See, e.g., In re Berg, 140 F.3d 1428, 46 USPQ2d 1226 (Fed. Cir. 1998); In re 
Goodman, 1 1 F.3d 1046, 29 USPQ2d 2010 (Fed. Cir. 1993); In re Longi, 759 F.2d 887, 225 
USPQ 645 (Fed. Cir. 1985); In re Van Ornum, 686 F.2d 937, 214 USPQ 761 (CCPA 1982); In re 
VogeU 422 F.2d 438, 164 USPQ 619 (CCPA 1970); and In re Thorington, 418 F.2d 528, 163 
USPQ 644 (CCPA 1969). 

A timely filed terminal disclaimer in compliance with 37 CFR 1.321(c) or 1.321(d) may 
be used to overcome an actual or provisional rejection based on a nonstatutory double patenting 
ground provided the conflicting application or patent either is shown to be commonly owned 
with this application, or claims an invention made as a result of activities undertaken within the 
scope of a joint research agreement. 

Effective January 1, 1994, a registered attorney or agent of record may sign a terminal 
disclaimer. A terminal disclaimer signed by the assignee must fully comply with 37 CFR 
3.73(b). 

12. Claims 19-32 are provisionally rejected on the ground of nonstatutory obviousness- 
type double patenting as being unpatentable over claims 6-10 and 12-18 of copending 
Application No. 11/415,096. Although the conflicting claims are not identical, they are not 
patentably distinct from each other because: 

Regarding independent claim(s) 19, every element of the claim is anticipated by claim 9 of the 
co-pending application (which incorporates its parent, claim 5), except the co-pending claim is 
the method that carried out by the apparatus of the instant claim. It would have been obvious to 
one of ordinary skill in the art at the time of the invention to use a computer (including a memory 
under control of a processor) to carry out the method of the co-pending claim, because a 
computer was one of several predictable finite solutions for implementing the method. 

The remainder of the instant application's claims correspond to the co-pending claims 
according to the table below, and differ from but are obvious from those claims for the same 
rationale as claim 19. 
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Instant 



Co-pending 



19 



20 



21 



This is a provisiona 



22 



8 



23 



24 



10 



25 



17 



26 



12 



27 



13 



28 



14 



29 



15 



30 



16 



31 



17 



32 



18 



obviousness-type double patenting rejection because the conflicting 
claims have not in fact been patented. 

13. Claims 19-32 are provisionally rejected on the ground of nonstatutory obviousness- 
type double patenting as being unpatentable over claims 16-20 and 24-30 of copending 
Application No. 10/948,316. Although the conflicting claims are not identical, they are not 
patentably distinct from each other because: 

Regarding independent claim(s) 19, every element of the claim is anticipated by claim 19 of 
the co-pending application (which incorporates its parent, claim 1 5), except the co-pending 
claims teach videos displayed in relation to audio, while the instant application teaches photos 
displayed in relation to audio. It would have been obvious to one of ordinary skill in the art at 
the time of the invention to display photos in relation to audio instead of video, as they were both 
a form of multimedia intended to be displayed in conjunction with other multimedia, and would 
have received the same benefit by displaying them according to a temporal relationship. 

The instant application's claims below correspond to the co-pending claims according to 
the table below, and differ from but are obvious from those claims for the same rationale as 
claim 19. 



Instant 
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20 
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24 


25 


26 
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28 


29 


30 


31 


32 


Co-pending 


19 


16 


17 


18 


19 


20 


29 


24 


25 


26 


27 


28 


29 


30 


This is a provisiona 


obviousness-type double patenting rejection because 



claims have not in fact been patented. 
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14. Claims 19-32 are provisionally rejected on the ground of nonstatutory obviousness- 
type double patenting as being unpatentable over claims 28, 30-34, 36-41 of copending 
Application No. 10/949,474. Although the conflicting claims are not identical, they are not 
patentably distinct from each other because: 

Regarding independent claim(s) 19, every element of the claim is anticipated by claim 28 of 
the co-pending application except the co-pending claims teach text displayed in relation to any 
multimedia, while the instant application teaches photos displayed in relation to audio. It would 
have been obvious to one of ordinary skill in the art at the time of the invention to display photos 
instead of text, in relation to audio instead of any multimedia, as they were all forms of 
multimedia intended to be displayed in conjunction with other multimedia, and would have 
received the same benefit by displaying them according to a temporal relationship. 

The instant application's claims below correspond to the co-pending claims according to 
the table below, and differ from but are obvious from those claims for the same rationale as 
claim 19. 



Instant 


19 


20 


21 


22 


23 


24 


25 


26 


27 


28 


29 


30 


31 


32 


Co-pending 


28 


30 


31 


32 


28 


33 


34 


36 


37 


38 


39 


40 


34 


41 



This is a provisional obviousness-type double patenting rejection because the conflicting claims 
have not in fact been patented. 

IS. Claims 19-32 are provisionally rejected on the ground of nonstatutory obviousness- 
type double patenting as being unpatentable over claims 18-22, and 25-31of copending 
Application No. 10/949,253. Although the conflicting claims are not identical, they are not 
patentably distinct from each other because: 
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Regarding independent claim(s) 19, every element of the claim is anticipated by claim 28 of 
the co-pending application except the co-pending claims teach photo displayed in relation to any 
video, while the instant application teaches photos displayed in relation to audio. It would have 
been obvious to one of ordinary skill in the art at the time of the invention to display photos in 
relation to audio instead of any video, as they were all forms of multimedia intended to be 
displayed in conjunction with other multimedia, and would have received the same benefit by 
displaying them according to a temporal relationship. 

The instant application's claims below correspond to the co-pending claims according to 
the table below, and differ from but are obvious from those claims for the same rationale as 
claim 19. 



Instant 


19 


20 


21 


22 


23 


24 


25 


26 


27 


28 


29 


30 


31 


32 


Co-pending 


21 


18 


19 


20 


21 


22 


30 


25 


26 


27 


28 


29 


30 


31 



This is a provisional obviousness-type double patenting rejection because the conflicting claims 
have not in fact been patented. 

Conclusion 

1 6. The prior art made of record and not relied upon is considered pertinent to applicant's 
disclosure. 

1 7. Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Adam M. Queler whose telephone number is (571) 272-4140. 
The examiner can normally be reached on Monday-Friday. 
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If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Steven Hong can be reached on (571) 272-4124. The fax phone number for the 
organization where this application or proceeding is assigned is 571-273-8300. 

Information regarding the status of an application may be obtained from the Patent 
Application Information Retrieval (PAIR) system. Status information for published applications 
may be obtained from either Private PAIR or Public PAIR. Status information for unpublished 
applications is available through Private PAIR only. For more information about the PAIR 
system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR 
system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would 
like assistance from a USPTO Customer Service Representative or access to the automated 
information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. 
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